Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: split read/write hotspot of hotspot_partition_calculator #592

Merged
merged 18 commits into from
Sep 17, 2020

Conversation

Smityz
Copy link
Contributor

@Smityz Smityz commented Aug 30, 2020

What problem does this PR solve?

This submission is mainly to change the data structure of historical partition hotspot statistical and split read/write hotspot of hotspot_partition_calculator

What is changed and how it works?

In the previous design, _partition_stat_histories saved in std::queue, which is a bad design.
in this PR, I will deprecate the formal data structure, change queue into list to save memory copy then split read/write hotpot of statistical, and add some unit tests by the way

File changes

src/server/hotspot_partition_data.h -> src/server/hotspot_partition_stat.h

@Smityz Smityz mentioned this pull request Aug 30, 2020
src/server/hotspot_algo_qps_variance.h Outdated Show resolved Hide resolved
src/server/hotspot_algo_qps_variance.cpp Outdated Show resolved Hide resolved
@Smityz Smityz marked this pull request as draft September 8, 2020 03:02
@Smityz Smityz marked this pull request as ready for review September 10, 2020 10:22
@Smityz Smityz changed the title refactor: split read/write hotkey of hotspot_policy refactor: split read/write hotkey of hotspot_partition_calculator Sep 10, 2020
@Smityz Smityz changed the title refactor: split read/write hotkey of hotspot_partition_calculator refactor: split read/write hotspot of hotspot_partition_calculator Sep 10, 2020
@Smityz Smityz marked this pull request as draft September 11, 2020 10:12
@Smityz Smityz marked this pull request as ready for review September 11, 2020 10:37
src/server/hotspot_partition_stat.h Outdated Show resolved Hide resolved
src/shell/command_helper.h Show resolved Hide resolved
src/server/hotspot_partition_calculator.h Outdated Show resolved Hide resolved
src/server/hotspot_partition_calculator.cpp Show resolved Hide resolved
src/server/hotspot_partition_calculator.h Show resolved Hide resolved
src/server/hotspot_partition_calculator.cpp Outdated Show resolved Hide resolved
src/server/hotspot_partition_calculator.cpp Outdated Show resolved Hide resolved
src/server/hotspot_partition_calculator.cpp Show resolved Hide resolved
src/server/hotspot_partition_calculator.cpp Outdated Show resolved Hide resolved
src/server/hotspot_partition_calculator.cpp Outdated Show resolved Hide resolved
src/server/hotspot_partition_stat.h Outdated Show resolved Hide resolved
}
}
}
standard_deviation = sqrt(standard_deviation / (sample_count - 1));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
standard_deviation = sqrt(standard_deviation / (sample_count - 1));
standard_deviation = sqrt(standard_deviation / (sample_count - 1));

(sample_count - 1) Why sub 1?

Copy link
Contributor Author

@Smityz Smityz Sep 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://en.wikipedia.org/wiki/Standard_deviation, the denominator in the sample standard deviation formula is N − 1.

src/server/hotspot_partition_calculator.h Outdated Show resolved Hide resolved
acelyc111
acelyc111 previously approved these changes Sep 15, 2020
acelyc111
acelyc111 previously approved these changes Sep 16, 2020
hycdong
hycdong previously approved these changes Sep 17, 2020
@Smityz Smityz dismissed stale reviews from hycdong and acelyc111 via 4cf88cc September 17, 2020 02:52
@acelyc111 acelyc111 merged commit 8697976 into apache:master Sep 17, 2020
@Smityz Smityz deleted the hotspot-2-formal branch September 18, 2020 02:08
@neverchanje neverchanje mentioned this pull request Mar 1, 2021
acelyc111 pushed a commit that referenced this pull request Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants